Key takeaways
Next Steps
The context: Why is this problem important to solve?
The objectives: What is the intended goal?
The key questions: What are the key questions that need to be answered?
The problem formulation: What is it that we are trying to solve using data science?
There are a total of 24,958 train and 2,600 test images (colored) that we have taken from microscopic images. These images are of the following categories:
Parasitized: The parasitized cells contain the Plasmodium parasite which causes malaria
Uninfected: The uninfected cells are free of the Plasmodium parasites
# Mount Google drive so that this notebook can be used from Colab and load the dataset from my Google drive
from google.colab import drive
drive.mount('/content/drive')
Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).
# Importing the required libraries
# The basic set is imported here, and we can import others as the need arises
import numpy as np
import pandas as pd
import tensorflow as tf
import matplotlib.pyplot as plt
import seaborn as sns
import zipfile
import io
from PIL import Image
# Suppress "will be deprecated in future" warning messages that we are ignoring knowingly
import warnings
warnings.filterwarnings("ignore", category=FutureWarning)
The data has been downloaded as a zip file using the link provided on Olympus
The zip file has different folders for train and test data and contain different sizes of images for parasitized and uninfected cells within the respective folders.
The size of all images must be the same and should be converted to 4D arrays so that they can be used as an input for the convolutional neural network. Also, we need to create the labels for both types of images to be able to train and test the model.
Let's do the same for the training data first and then we will use the same code for the test data as well.
# Create a re-usable function to prepare image datasets
# We will read image files directly from the zip file
# Will read both parasitized and uninfected images under the path passed in
# Will return the images dataset in 4D, ready to use in our models
# And will return the labels in 1D
def prepare_images_dataset(zip_archive, # zip file that is already open for reading
file_path_prefix): # path prefix for train / test datasets
# initialize the datatsets to be returned
X = []
y = []
# load the images
for filename in zip_archive.namelist():
if filename.startswith((file_path_prefix)): # only process files under file_path_prefix directory
if filename.endswith(('.png')): # ignore non-image files
with zip_archive.open(filename) as image_file:
# read the image and add to the list in correct format
image = Image.open(io.BytesIO(image_file.read()))
image = image.convert('RGB') # Convert them to RGB format
resized_image = image.resize((64, 64)) # Resize images to a standard size, as images of different sizes can't be used in models
npImage = np.array(resized_image)
X.append(npImage)
if filename.startswith((file_path_prefix + 'parasitized/')):
y.append(1) # the image is of a parasitized cell
else:
y.append(0) # the image is of an uninfected cell
# Convert the list to arrays
X = np.array(X)
y = np.array(y)
return X, y # these datasets are ready to be used in our models
# Load images and labels for both train and test datasets
# Use one of the two lines below to open the zip file with images, depending on if you are running the notebook locally or on Colab
# zip_archive = zipfile.ZipFile('cell_images.zip', 'r')
zip_archive = zipfile.ZipFile('/content/drive/MyDrive/mit-adsp/Capstone/cell_images.zip', 'r')
X_train, y_train = prepare_images_dataset(zip_archive, 'cell_images/train/')
X_test, y_test = prepare_images_dataset(zip_archive, 'cell_images/test/')
zip_archive.close() # close the zip file, that we don't need it anymore
del zip_archive # free-up the memory used by zip_archive
# Let's printout the shapes for image datasets
print('Shape of training dataset :', X_train.shape)
print('Shape of test dataset :', X_test.shape)
Shape of training dataset : (24958, 64, 64, 3) Shape of test dataset : (2600, 64, 64, 3)
# Printout the shpaes for lable arrays
print('Shape of training labels :', y_train.shape)
print('Shape of test labels :', y_test.shape)
Shape of training labels : (24958,) Shape of test labels : (2600,)
# Print the range of pixel values, from minimum to maximum, for each dataset
print('The range (min - max) of pixel values for training images :', np.amin(X_train), ' - ', np.amax(X_train))
print('The range (min - max) of pixel values for test images :', np.amin(X_test), ' - ', np.amax(X_test))
The range (min - max) of pixel values for training images : 0 - 255 The range (min - max) of pixel values for test images : 0 - 255
# Get the count of uninfected and parasitized cell images in the training set
values, counts = np.unique(y_train, return_counts=True)
print('Training set: Number of uninfected cell images :', counts[0])
print('Training set: Number of parasitized cell images :', counts[1])
# Get the count of uninfected and parasitized cell images in the test set
values, counts = np.unique(y_test, return_counts=True)
print('\nTesting set: Number of uninfected cell images :', counts[0])
print('Testing set: Number of parasitized cell images :', counts[1])
Training set: Number of uninfected cell images : 12376 Training set: Number of parasitized cell images : 12582 Testing set: Number of uninfected cell images : 1300 Testing set: Number of parasitized cell images : 1300
# As we have seen pixel values in the training and test dataset range between 0 and 255
# Since the training would be efficient when we deal with normalized data, let's normalize both train and test datasets
X_train_normalized = X_train.astype('float32') / 255.0
X_test_normalized = X_test.astype('float32') / 255.0
# Print out the range of values after normalization
print('The normalized range (min - max) of pixel values for training images :', np.amin(X_train_normalized), ' - ', np.amax(X_train_normalized))
print('The normalized range (min - max) of pixel values for test images :', np.amin(X_test_normalized), ' - ', np.amax(X_test_normalized))
The normalized range (min - max) of pixel values for training images : 0.0 - 1.0 The normalized range (min - max) of pixel values for test images : 0.0 - 1.0
# Let's use a histogram to visualize if parasitized and unaffected datasets are balanced, for both train and test datasets
plt.figure(figsize=(12, 6))
# plot the histograms side by side for both datasets, train dataset first
plt.subplot(1, 2, 1)
plt.hist(y_train, bins=2, rwidth=0.60 )
plt.title('Frequency Distribution of Training set')
plt.xticks([0, 1])
plt.xlabel('Parasitized?')
plt.ylabel('Image count')
# plot the histogram for test dataset
plt.subplot(1, 2, 2)
plt.hist(y_test, bins=2, rwidth=0.60 )
plt.title('Frequency Distribution of Test set')
plt.xticks([0, 1])
plt.xlabel('Parasitized?')
plt.ylabel('Image count')
plt.show()
# Seed the random numbers first
import random
random.seed(9)
np.random.seed(9)
tf.random.set_seed(9)
# Create a function to display a random number of images from the given dataset
def displayImages(dataset, # dataset of images to be used
rows, # number of rows in the plot
cols, # number of columns in the plot
figsize, # Size of the figure
showLables): # Whether to show labels or not
labels = ['Uninfected', 'Parasitized'] # labels to show for images
tmp_array = np.arange(dataset.shape[0]-1) # random index values to choose from, based on image count in training set
images_to_show = np.random.choice(tmp_array, size = rows * cols, replace=False)
fig = plt.figure(figsize=figsize) # Defining the figure size to 12x6
for i in range(rows): # Iterate through rows first
for j in range(cols): # For each row, iterate through columns
index = i * cols + j # Calculating the index for the image to plot
ax = fig.add_subplot(rows, cols, index + 1) # Adding subplot
ax.imshow(dataset[images_to_show[index], :], # Plotting the image using cmap=gray
cmap=plt.get_cmap('gray'))
if (showLables):
ax.set_title(labels[y_train[images_to_show[index]]] + # Show the labels as the caption for images
' [' + str(images_to_show[index]) + ']', fontsize=9)
plt.xticks([]) # Remove un-necessary ticks on the X axis
plt.yticks([]) # Remove un-necessary ticks on the y axis
plt.show() # Finally, display the plot
# Display a random 10 images from the training dataset along with their labels
displayImages(X_train_normalized, 2, 5, (12, 6), True)
# Display a random 36 images in a 6x6 plot as requested from the training dataset along with their labels
displayImages(X_train_normalized, 6, 6, (12, 12), True)
# Extract parasitied images into a separate set and visualize some random samples
X_train_parasitized = X_train_normalized[y_train == 1]
displayImages(X_train_parasitized, 3, 6, (12, 6), False)
# Extract uninfected images into a separate set and visualize some random samples
X_train_uninfected = X_train_normalized[y_train == 0]
displayImages(X_train_uninfected, 3, 6, (12, 6), False)
# Let's calculate the mean images for parasitized and uninfected sets from the training dataset
mean_parasitized_image = np.mean(X_train_parasitized, axis=0)
mean_uninfected_image = np.mean(X_train_uninfected, axis=0)
print(mean_parasitized_image.shape, mean_uninfected_image.shape)
(64, 64, 3) (64, 64, 3)
Mean image for parasitized
# display the mean parasitized image
plt.imshow(mean_parasitized_image, cmap=plt.get_cmap('gray'))
plt.show()
Mean image for uninfected
# display the mean uninfected image
plt.imshow(mean_uninfected_image, cmap=plt.get_cmap('gray'))
plt.show()
from sklearn.preprocessing import MinMaxScaler
def DisplayMeanAndDifferenceImages(sample_image, mean_image, image_type):
diff_image = sample_image - mean_image
# As the difference image has negative values due to subtraction, we need to rescale them between 0 and 1
# Using Normalize in imshow does not seem to help, so explicitly rescaling the image
scaler = MinMaxScaler(feature_range=(0, 1))
diff_image_2d = diff_image.reshape(-1, 1)
diff_image_scleaed_2d = scaler.fit_transform(diff_image_2d)
diff_image_scaled = diff_image_scleaed_2d.reshape(diff_image.shape)
fig, axs = plt.subplots(1, 3, figsize=(12, 6))
axs[0].imshow(sample_image)
axs[0].set_title('Sample Image - ' + image_type)
axs[1].imshow(mean_image)
axs[1].set_title('Mean Image - ' + image_type)
axs[2].imshow(diff_image_scaled)
axs[2].set_title('Difference Image - ' + image_type)
plt.show()
# Display a random parasitized sample image and its difference image from its mean
img_to_display = np.random.randint(0, X_train_parasitized.shape[0]) # choose a random sample
DisplayMeanAndDifferenceImages(X_train_parasitized[img_to_display], mean_parasitized_image, 'Parasitized')
np.random.randint(0, X_train_parasitized.shape[0])
3995
# Display a random uninfected sample image and its difference image from its mean
img_to_display = np.random.randint(0, X_train_uninfected.shape[0]) # choose a random sample
DisplayMeanAndDifferenceImages(X_train_uninfected[img_to_display], mean_uninfected_image, 'Uninfected')
# Convert the training dataset images to HSV color space
import cv2
X_train_hsv = []
# convert the original un-normalized RGB images to HSV colorspace
for rgb_image in X_train:
hsv_image = cv2.cvtColor(rgb_image, cv2.COLOR_RGB2HSV)
X_train_hsv.append(hsv_image)
X_train_hsv = np.array(X_train_hsv)
# normalize the hsv images so that we can utilize them in our models
X_train_normalized_hsv = X_train_hsv.astype('float32') / 255.0
print(X_train_normalized_hsv.shape)
(24958, 64, 64, 3)
# we will pick random 18 images from the dataset, and display their original RGB and processed images side by side
# so that we can compare them easily
def displayOriginalAndProcessedImages(rgb_images, processed_images):
labels = ['Uninfected', 'Parasitized']
tmp_array = np.arange(rgb_images.shape[0])
images_to_show = np.random.choice(tmp_array, size=18, replace=False)
fig = plt.figure(figsize=(12, 12))
rows = 6 # total of 36 images (18x2)
cols = 6
img_to_show_index = 0
for i in range(18):
index = i * 2 + 1
# Show RGB image first
ax = fig.add_subplot(rows, cols, index)
ax.imshow(rgb_images[images_to_show[i], :], cmap=plt.get_cmap('gray'))
ax.set_title(labels[y_train[images_to_show[i]]] + ' [orig-' + str(images_to_show[i]) + ']', fontsize=7)
plt.xticks([])
plt.yticks([])
# Show HSV image next to it
ax = fig.add_subplot(rows, cols, index + 1)
ax.imshow(processed_images[images_to_show[i], :], cmap=plt.get_cmap('gray'))
ax.set_title(labels[y_train[images_to_show[i]]] + ' [proc-' + str(images_to_show[i]) + ']', fontsize=7)
plt.xticks([])
plt.yticks([])
plt.show()
# Compare a random set of RGB and their corresponging HSV images from the training dataset
displayOriginalAndProcessedImages(X_train_normalized, X_train_normalized_hsv)
# Convert the test dataset images to HSV color space
X_test_hsv = []
for rgb_image in X_test:
hsv_image = cv2.cvtColor(rgb_image, cv2.COLOR_RGB2HSV)
X_test_hsv.append(hsv_image)
X_test_hsv = np.array(X_test_hsv)
# normalize the hsv images so that we can utilize them in our models
X_test_normalized_hsv = X_test_hsv.astype('float32') / 255.0
print(X_test_normalized_hsv.shape)
(2600, 64, 64, 3)
# Compare a random set of RGB and their corresponging HSV images from the test dataset, visually
displayOriginalAndProcessedImages(X_test_normalized, X_test_normalized_hsv)
# Let's build the Gausian blurred images for training set
X_train_normalized_blurred = []
for image in X_train_normalized:
blurred_image = cv2.GaussianBlur(image, (5, 5), 0)
X_train_normalized_blurred.append(blurred_image)
X_train_normalized_blurred = np.array(X_train_normalized_blurred)
print(X_train_normalized_blurred.shape)
(24958, 64, 64, 3)
# Compare a random set of RGB and their corresponging Gaussain blurred images from the training dataset, visually
displayOriginalAndProcessedImages(X_train_normalized, X_train_normalized_blurred)
# Let's build the Gausian blurred images for test set
X_test_normalized_blurred = []
for image in X_test_normalized:
blurred_image = cv2.GaussianBlur(image, (5, 5), 0)
X_test_normalized_blurred.append(blurred_image)
X_test_normalized_blurred = np.array(X_test_normalized_blurred)
print(X_test_normalized_blurred.shape)
(2600, 64, 64, 3)
# Compare a random set of RGB and their corresponging Gaussain blurred images from the test dataset, visually
displayOriginalAndProcessedImages(X_test_normalized, X_test_normalized_blurred)
Think About It: Would blurring help us for this problem statement in any way? What else can we try?
Note: The Base Model should be fully built and evaluated with all outputs shown to give an idea about the process of the creation and evaluation of the performance of a CNN architecture. A similar process can be followed in iterating to build better-performing CNN architectures.
# Import libararies needed for ANN
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPooling2D, BatchNormalization, LeakyReLU
from keras.optimizers import Adam
# we need to one-hot encode labels so that we can use them in the ANN/CNN models
y_train_encoded = tf.keras.utils.to_categorical(y_train)
y_test_encoded = tf.keras.utils.to_categorical(y_test)
# Re-init random numbers
random.seed(3)
np.random.seed(3)
tf.random.set_seed(3)
X_train_normalized.shape
(24958, 64, 64, 3)
# Create a base CNN model just to showcase the process involved in creating and evaluating the performance of a CNN model
model_base = Sequential()
model_base.add(Conv2D(16, (3, 3), padding="same", activation='relu', input_shape=(64, 64, 3))) # input images are of dimension (64, 64, 3)
model_base.add(MaxPooling2D(pool_size=(2, 2)))
model_base.add(Flatten())
model_base.add(Dense(32, activation='relu'))
model_base.add(Dense(2, activation='softmax'))
# Compile the model, using Adam optimizer and accuracy for metrics
model_base.compile(loss='categorical_crossentropy', optimizer=Adam(learning_rate=0.001), metrics=['accuracy'])
# Let's define a callback to enable early stopping, if no improvement between successive epochs
early_stopping_cb = tf.keras.callbacks.EarlyStopping(
monitor='val_accuracy', min_delta=0.05, patience=3, verbose=1, restore_best_weights=False)
# Train the model using trainiing set of normalized images with the early stopping callback
# Let's use 20% of data for validation in each epoch
model_history = model_base.fit(X_train_normalized, y_train_encoded, validation_split=0.2, batch_size=32, verbose=1,
epochs=50, callbacks=[early_stopping_cb])
Epoch 1/50 624/624 [==============================] - 6s 5ms/step - loss: 0.5586 - accuracy: 0.7160 - val_loss: 0.7892 - val_accuracy: 0.6062 Epoch 2/50 624/624 [==============================] - 3s 4ms/step - loss: 0.3633 - accuracy: 0.8573 - val_loss: 0.5340 - val_accuracy: 0.7919 Epoch 3/50 624/624 [==============================] - 2s 4ms/step - loss: 0.2364 - accuracy: 0.9123 - val_loss: 0.6392 - val_accuracy: 0.7582 Epoch 4/50 624/624 [==============================] - 2s 4ms/step - loss: 0.1802 - accuracy: 0.9342 - val_loss: 0.3798 - val_accuracy: 0.8486 Epoch 5/50 624/624 [==============================] - 2s 4ms/step - loss: 0.1533 - accuracy: 0.9441 - val_loss: 0.4224 - val_accuracy: 0.8566 Epoch 6/50 624/624 [==============================] - 2s 4ms/step - loss: 0.1269 - accuracy: 0.9557 - val_loss: 0.3219 - val_accuracy: 0.8844 Epoch 7/50 624/624 [==============================] - 3s 4ms/step - loss: 0.1179 - accuracy: 0.9586 - val_loss: 0.2723 - val_accuracy: 0.9141 Epoch 8/50 624/624 [==============================] - 2s 4ms/step - loss: 0.0875 - accuracy: 0.9708 - val_loss: 0.3540 - val_accuracy: 0.8802 Epoch 9/50 624/624 [==============================] - 2s 4ms/step - loss: 0.0739 - accuracy: 0.9735 - val_loss: 0.2889 - val_accuracy: 0.9002 Epoch 10/50 624/624 [==============================] - 2s 4ms/step - loss: 0.0605 - accuracy: 0.9790 - val_loss: 0.7237 - val_accuracy: 0.7943 Epoch 10: early stopping
# Let's evaluate the model for accuracy against test dataset
test_accuracy = model_base.evaluate(X_test_normalized, y_test_encoded, verbose=2)
82/82 - 0s - loss: 0.4066 - accuracy: 0.8692 - 319ms/epoch - 4ms/step
# Predict the labels for the test dataset
y_pred = model_base.predict(X_test_normalized) # returns prediction probabilities for each label
# Convert the prediction probablities back to single labels
y_pred_labels = np.argmax(y_pred, axis=1)
y_test_labels = np.argmax(y_test_encoded, axis=1)
82/82 [==============================] - 0s 2ms/step
# Print the Classification report
from sklearn.metrics import classification_report
metrics = classification_report(y_test_labels, y_pred_labels)
print(metrics)
precision recall f1-score support
0 0.95 0.78 0.86 1300
1 0.81 0.96 0.88 1300
accuracy 0.87 2600
macro avg 0.88 0.87 0.87 2600
weighted avg 0.88 0.87 0.87 2600
# Print the Confusion Matrix for test predictions using base CNN model
confusion_matrix = tf.math.confusion_matrix(y_test_labels, y_pred_labels)
_, ax = plt.subplots(figsize=(5, 4))
sns.heatmap(confusion_matrix, annot=True, linewidths=.4, fmt="d", square=True, ax=ax)
plt.show()
# Collect the performance metrics into a dataframe for a later comparison of various models
from sklearn.metrics import precision_recall_fscore_support
model_performances = pd.DataFrame(columns=['Model', 'Accuracy',
'Precision Parasitized', 'Precision Uninfected',
'Recall Parasitized', 'Recall Uninfected',
'F1 Score Parasitized', 'F1 Score Uninfected'])
precision, recall, f1_score, _ = precision_recall_fscore_support(y_test_labels, y_pred_labels)
accuracy = float(metrics.split()[-2])
model_performances = model_performances.append({
'Model' : 'Base CNN Model',
'Accuracy' : accuracy,
'Precision Parasitized' : round(precision[1], 2),
'Precision Uninfected' : round(precision[0], 2),
'Recall Parasitized' : round(recall[1], 2),
'Recall Uninfected' : round(recall[0], 2),
'F1 Score Parasitized' : round(f1_score[1], 2),
'F1 Score Uninfected' : round(f1_score[0], 2)
}, ignore_index=True)
# Check that we are capturing the performance metrics properly
model_performances
| Model | Accuracy | Precision Parasitized | Precision Uninfected | Recall Parasitized | Recall Uninfected | F1 Score Parasitized | F1 Score Uninfected | |
|---|---|---|---|---|---|---|---|---|
| 0 | Base CNN Model | 0.87 | 0.81 | 0.95 | 0.96 | 0.78 | 0.88 | 0.86 |
# Plot the accuracy of training and validation sets from the model_history that is passed in
def plotAccuracy(model_history, model_name):
plt.plot(model_history.history['accuracy'])
plt.plot(model_history.history['val_accuracy'])
plt.title('Accuracy of : ' + model_name)
plt.ylabel('Accuracy')
plt.xlabel('Epoch Count')
plt.legend(['Training Set Performance', 'Validation Set Performance'], loc='lower right')
plt.show()
# Plot the accuracy of training and validation sets for the base CNN model
plotAccuracy(model_history, 'Base CNN Model')
So now let's try to build another model with few more add on layers and try to check if we can try to improve the model. Therefore try to build a model by adding few layers if required and altering the activation functions.
Trying to improve the performance of our model by adding new layers
# Clear Keras' backend to clear the history of previous models
from keras import backend as keras_backend
keras_backend.clear_session()
# Re-init random numbers
random.seed(6)
np.random.seed(6)
tf.random.set_seed(6)
# Add additional layers of Conv2D with LeakyRelu and Dense(16) layers
model_more_layers = Sequential()
model_more_layers.add(Conv2D(16, (3, 3), padding="same", activation='relu', input_shape=(64, 64, 3)))
model_more_layers.add(Conv2D(32, (3, 3), padding="same"))
model_more_layers.add(LeakyReLU(alpha=0.1))
model_more_layers.add(MaxPooling2D(pool_size=(2, 2)))
model_more_layers.add(Flatten())
model_more_layers.add(Dense(32, activation='relu'))
model_more_layers.add(Dense(16, activation='relu'))
model_more_layers.add(Dense(2, activation='softmax'))
# Compile the CNN model with more layers
model_more_layers.compile(loss='categorical_crossentropy', optimizer=Adam(learning_rate=0.001), metrics=['accuracy'])
# Let's define a callback to enable early stopping, if no improvement between successive epochs
early_stopping_cb = tf.keras.callbacks.EarlyStopping(
monitor='val_accuracy', min_delta=0.05, patience=5, verbose=1, restore_best_weights=True)
# Train the model with more layers using training dataset and early stopping callback
model_history = model_more_layers.fit(X_train_normalized, y_train_encoded, validation_split=0.2, batch_size=32, verbose=1,
epochs=50, callbacks=[early_stopping_cb])
Epoch 1/50 624/624 [==============================] - 7s 7ms/step - loss: 0.4835 - accuracy: 0.7569 - val_loss: 0.6069 - val_accuracy: 0.6879 Epoch 2/50 624/624 [==============================] - 4s 6ms/step - loss: 0.1972 - accuracy: 0.9238 - val_loss: 0.4102 - val_accuracy: 0.8834 Epoch 3/50 624/624 [==============================] - 4s 6ms/step - loss: 0.1491 - accuracy: 0.9455 - val_loss: 0.3377 - val_accuracy: 0.8704 Epoch 4/50 624/624 [==============================] - 4s 6ms/step - loss: 0.1253 - accuracy: 0.9540 - val_loss: 0.1671 - val_accuracy: 0.9223 Epoch 5/50 624/624 [==============================] - 4s 6ms/step - loss: 0.1230 - accuracy: 0.9538 - val_loss: 0.1907 - val_accuracy: 0.9287 Epoch 6/50 624/624 [==============================] - 4s 6ms/step - loss: 0.0901 - accuracy: 0.9666 - val_loss: 0.1936 - val_accuracy: 0.9411 Epoch 7/50 624/624 [==============================] - 4s 6ms/step - loss: 0.0746 - accuracy: 0.9723 - val_loss: 0.1393 - val_accuracy: 0.9531 Epoch 8/50 624/624 [==============================] - 4s 6ms/step - loss: 0.0638 - accuracy: 0.9764 - val_loss: 0.2543 - val_accuracy: 0.9000 Epoch 9/50 624/624 [==============================] - 4s 6ms/step - loss: 0.0537 - accuracy: 0.9801 - val_loss: 0.3777 - val_accuracy: 0.9281 Epoch 10/50 624/624 [==============================] - 4s 6ms/step - loss: 0.0360 - accuracy: 0.9867 - val_loss: 0.1612 - val_accuracy: 0.9601 Epoch 11/50 623/624 [============================>.] - ETA: 0s - loss: 0.0366 - accuracy: 0.9874Restoring model weights from the end of the best epoch: 6. 624/624 [==============================] - 4s 6ms/step - loss: 0.0367 - accuracy: 0.9874 - val_loss: 0.2140 - val_accuracy: 0.9447 Epoch 11: early stopping
# Let's evaluate the model with more layers for accuracy against test dataset
test_accuracy = model_more_layers.evaluate(X_test_normalized, y_test_encoded, verbose=2)
82/82 - 0s - loss: 0.1826 - accuracy: 0.9396 - 391ms/epoch - 5ms/step
# Predict the labels for the test dataset
y_pred = model_more_layers.predict(X_test_normalized) # returns prediction probabilities for each label
# Convert the prediction probablities back to single labels
y_pred_labels = np.argmax(y_pred, axis=1)
y_test_labels = np.argmax(y_test_encoded, axis=1)
82/82 [==============================] - 0s 2ms/step
# Print the Classification report
metrics = classification_report(y_test_labels, y_pred_labels)
print(metrics)
precision recall f1-score support
0 0.93 0.95 0.94 1300
1 0.95 0.93 0.94 1300
accuracy 0.94 2600
macro avg 0.94 0.94 0.94 2600
weighted avg 0.94 0.94 0.94 2600
# Print the Confusion Matrix for test predictions
confusion_matrix = tf.math.confusion_matrix(y_test_labels, y_pred_labels)
_, ax = plt.subplots(figsize=(5, 4))
sns.heatmap(confusion_matrix, annot=True, linewidths=.4, fmt="d", square=True, ax=ax)
plt.show()
# Plot the accuracy of training and validation sets for the model with more layers
plotAccuracy(model_history, 'Model with more layers')
# Collect the performance metrics for a later comparison of various models
precision, recall, f1_score, _ = precision_recall_fscore_support(y_test_labels, y_pred_labels)
accuracy = float(metrics.split()[-2])
model_performances = model_performances.append({
'Model' : 'Using more layers',
'Accuracy' : accuracy,
'Precision Parasitized' : round(precision[1], 2),
'Precision Uninfected' : round(precision[0], 2),
'Recall Parasitized' : round(recall[1], 2),
'Recall Uninfected' : round(recall[0], 2),
'F1 Score Parasitized' : round(f1_score[1], 2),
'F1 Score Uninfected' : round(f1_score[0], 2)
}, ignore_index=True)
# Let's look at the performances of models we have created so far
model_performances
| Model | Accuracy | Precision Parasitized | Precision Uninfected | Recall Parasitized | Recall Uninfected | F1 Score Parasitized | F1 Score Uninfected | |
|---|---|---|---|---|---|---|---|---|
| 0 | Base CNN Model | 0.87 | 0.81 | 0.95 | 0.96 | 0.78 | 0.88 | 0.86 |
| 1 | Using more layers | 0.94 | 0.95 | 0.93 | 0.93 | 0.95 | 0.94 | 0.94 |
Now let's build a model with LeakyRelu as the activation function
Let us try to build a model using BatchNormalization and using LeakyRelu as our activation function.
# Clear Keras' backend to clear the history of previous models
keras_backend.clear_session()
# Re-init random numbers
random.seed(9)
np.random.seed(9)
tf.random.set_seed(9)
# Create a more complex model with additional layers including BatchNormalization layers, and use LeakyRelu as the activation function
model_batch_norm = Sequential()
model_batch_norm.add(Conv2D(16, (3, 3), padding="same", input_shape=(64, 64, 3)))
model_batch_norm.add(LeakyReLU(alpha=0.1))
model_batch_norm.add(Conv2D(32, (3, 3), padding="same"))
model_batch_norm.add(LeakyReLU(alpha=0.1))
model_batch_norm.add(MaxPooling2D(pool_size=(2, 2)))
model_batch_norm.add(BatchNormalization())
model_batch_norm.add(Conv2D(32, (3, 3), padding="same"))
model_batch_norm.add(LeakyReLU(alpha=0.1))
model_batch_norm.add(Conv2D(64, (3, 3), padding="same"))
model_batch_norm.add(LeakyReLU(alpha=0.1))
model_batch_norm.add(MaxPooling2D(pool_size=(2, 2)))
model_batch_norm.add(BatchNormalization())
model_batch_norm.add(Flatten())
model_batch_norm.add(Dense(32))
model_batch_norm.add(LeakyReLU(alpha=0.1))
model_batch_norm.add(Dropout(0.5))
model_batch_norm.add(Dense(2, activation='softmax'))
# Compile the model with BatchNormalization and using LeakyRelu activation
model_batch_norm.compile(loss='categorical_crossentropy', optimizer=Adam(learning_rate=0.001), metrics=['accuracy'])
# Let's define a callback to enable early stopping, if no improvement between successive epochs
early_stopping_cb = tf.keras.callbacks.EarlyStopping(
monitor='val_accuracy', min_delta=0.05, patience=3, verbose=1, restore_best_weights=True)
# Train the model with BatchNormalization and LeakyRelu using training dataset and early stopping callback
model_history = model_batch_norm.fit(X_train_normalized, y_train_encoded, validation_split=0.2, batch_size=32, verbose=1,
epochs=50, callbacks=[early_stopping_cb])
Epoch 1/50 624/624 [==============================] - 9s 10ms/step - loss: 0.4434 - accuracy: 0.8185 - val_loss: 0.0167 - val_accuracy: 0.9968 Epoch 2/50 624/624 [==============================] - 6s 9ms/step - loss: 0.1313 - accuracy: 0.9562 - val_loss: 0.0945 - val_accuracy: 0.9898 Epoch 3/50 624/624 [==============================] - 6s 9ms/step - loss: 0.1053 - accuracy: 0.9676 - val_loss: 0.0999 - val_accuracy: 0.9740 Epoch 4/50 619/624 [============================>.] - ETA: 0s - loss: 0.0831 - accuracy: 0.9720Restoring model weights from the end of the best epoch: 1. 624/624 [==============================] - 6s 9ms/step - loss: 0.0828 - accuracy: 0.9721 - val_loss: 0.1908 - val_accuracy: 0.9459 Epoch 4: early stopping
# Let's evaluate the model for accuracy against test dataset
test_accuracy = model_batch_norm.evaluate(X_test_normalized, y_test_encoded, verbose=2)
82/82 - 1s - loss: 0.4548 - accuracy: 0.8719 - 509ms/epoch - 6ms/step
# Predict the labels for the test dataset
y_pred = model_batch_norm.predict(X_test_normalized) # returns prediction probabilities for each label
# Convert the prediction probablities back to single labels
y_pred_labels = np.argmax(y_pred, axis=1)
y_test_labels = np.argmax(y_test_encoded, axis=1)
metrics = classification_report(y_test_labels, y_pred_labels)
print(metrics)
82/82 [==============================] - 0s 3ms/step
precision recall f1-score support
0 0.80 1.00 0.89 1300
1 0.99 0.75 0.85 1300
accuracy 0.87 2600
macro avg 0.90 0.87 0.87 2600
weighted avg 0.90 0.87 0.87 2600
# Print the Confusion Matrix for test predictions
confusion_matrix = tf.math.confusion_matrix(y_test_labels, y_pred_labels)
_, ax = plt.subplots(figsize=(5, 4))
sns.heatmap(confusion_matrix, annot=True, linewidths=.4, fmt="d", square=True, ax=ax)
plt.show()
# Plot the accuracy of training and validation sets for the model with more layers
plotAccuracy(model_history, 'Model with BatchNormalization and LeakyRelu')
# Collect the performance metrics for a later comparison of various models
precision, recall, f1_score, _ = precision_recall_fscore_support(y_test_labels, y_pred_labels)
accuracy = float(metrics.split()[-2])
model_performances = model_performances.append({
'Model' : 'Using Batch Normalization',
'Accuracy' : accuracy,
'Precision Parasitized' : round(precision[1], 2),
'Precision Uninfected' : round(precision[0], 2),
'Recall Parasitized' : round(recall[1], 2),
'Recall Uninfected' : round(recall[0], 2),
'F1 Score Parasitized' : round(f1_score[1], 2),
'F1 Score Uninfected' : round(f1_score[0], 2)
}, ignore_index=True)
# print out the performances of models created so far
model_performances
| Model | Accuracy | Precision Parasitized | Precision Uninfected | Recall Parasitized | Recall Uninfected | F1 Score Parasitized | F1 Score Uninfected | |
|---|---|---|---|---|---|---|---|---|
| 0 | Base CNN Model | 0.87 | 0.81 | 0.95 | 0.96 | 0.78 | 0.88 | 0.86 |
| 1 | Using more layers | 0.94 | 0.95 | 0.93 | 0.93 | 0.95 | 0.94 | 0.94 |
| 2 | Using Batch Normalization | 0.87 | 0.99 | 0.80 | 0.75 | 1.00 | 0.85 | 0.89 |
We saw earlier that HSV images seemed to visually highlight the patterns better. Let us try using HSV images to train a similar model
# Clear Keras' backend to clear the history of previous models
keras_backend.clear_session()
# Re-init random numbers
random.seed(12)
np.random.seed(12)
tf.random.set_seed(12)
# Create a similar model as the last one, with additional layers including BatchNormalization layers, and use LeakyRelu as the activation function
model_batch_norm_hsv = Sequential()
model_batch_norm_hsv.add(Conv2D(16, (3, 3), padding="same", input_shape=(64, 64, 3)))
model_batch_norm_hsv.add(LeakyReLU(alpha=0.1))
model_batch_norm_hsv.add(Conv2D(32, (3, 3), padding="same"))
model_batch_norm_hsv.add(LeakyReLU(alpha=0.1))
model_batch_norm_hsv.add(MaxPooling2D(pool_size=(2, 2)))
model_batch_norm_hsv.add(BatchNormalization())
model_batch_norm_hsv.add(Conv2D(32, (3, 3), padding="same"))
model_batch_norm_hsv.add(LeakyReLU(alpha=0.1))
model_batch_norm_hsv.add(Conv2D(64, (3, 3), padding="same"))
model_batch_norm_hsv.add(LeakyReLU(alpha=0.1))
model_batch_norm_hsv.add(MaxPooling2D(pool_size=(2, 2)))
model_batch_norm_hsv.add(BatchNormalization())
model_batch_norm_hsv.add(Flatten())
model_batch_norm_hsv.add(Dense(32))
model_batch_norm_hsv.add(LeakyReLU(alpha=0.1))
model_batch_norm_hsv.add(Dropout(0.5))
model_batch_norm_hsv.add(Dense(2, activation='softmax'))
# Compile the model with BatchNormalization and using LeakyRelu activation
model_batch_norm_hsv.compile(loss='categorical_crossentropy', optimizer=Adam(learning_rate=0.001), metrics=['accuracy'])
# Train the model with BatchNormalization and LeakyRelu using training dataset and early stopping callback
model_history = model_batch_norm_hsv.fit(X_train_normalized_hsv, y_train_encoded, validation_split=0.2, batch_size=32, verbose=1,
epochs=50, callbacks=[early_stopping_cb])
Epoch 1/50 624/624 [==============================] - 9s 10ms/step - loss: 0.3870 - accuracy: 0.8598 - val_loss: 0.3046 - val_accuracy: 0.9573 Epoch 2/50 624/624 [==============================] - 6s 9ms/step - loss: 0.1371 - accuracy: 0.9577 - val_loss: 0.0241 - val_accuracy: 0.9944 Epoch 3/50 624/624 [==============================] - 6s 9ms/step - loss: 0.1168 - accuracy: 0.9645 - val_loss: 0.2881 - val_accuracy: 0.9513 Epoch 4/50 623/624 [============================>.] - ETA: 0s - loss: 0.0915 - accuracy: 0.9725Restoring model weights from the end of the best epoch: 1. 624/624 [==============================] - 6s 9ms/step - loss: 0.0914 - accuracy: 0.9725 - val_loss: 0.3137 - val_accuracy: 0.9361 Epoch 4: early stopping
# Let's evaluate the model for accuracy against test dataset
test_accuracy = model_batch_norm_hsv.evaluate(X_test_normalized_hsv, y_test_encoded, verbose=2)
82/82 - 0s - loss: 0.2903 - accuracy: 0.9408 - 457ms/epoch - 6ms/step
# Predict the labels for the test dataset
y_pred = model_batch_norm_hsv.predict(X_test_normalized_hsv) # returns prediction probabilities for each label
# Convert the prediction probablities back to single labels
y_pred_labels = np.argmax(y_pred, axis=1)
y_test_labels = np.argmax(y_test_encoded, axis=1)
metrics = classification_report(y_test_labels, y_pred_labels)
print(metrics)
82/82 [==============================] - 0s 3ms/step
precision recall f1-score support
0 0.94 0.94 0.94 1300
1 0.94 0.94 0.94 1300
accuracy 0.94 2600
macro avg 0.94 0.94 0.94 2600
weighted avg 0.94 0.94 0.94 2600
# Print the Confusion Matrix for test predictions
confusion_matrix = tf.math.confusion_matrix(y_test_labels, y_pred_labels)
_, ax = plt.subplots(figsize=(5, 4))
sns.heatmap(confusion_matrix, annot=True, linewidths=.4, fmt="d", square=True, ax=ax)
plt.show()
# Plot the accuracy of training and validation sets for the model with more layers
plotAccuracy(model_history, 'Model with BatchNormalization and LeakyRelu that uses HSV images')
# Collect the performance metrics for a later comparison of various models
precision, recall, f1_score, _ = precision_recall_fscore_support(y_test_labels, y_pred_labels)
accuracy = float(metrics.split()[-2])
model_performances = model_performances.append({
'Model' : 'Using HSV images',
'Accuracy' : accuracy,
'Precision Parasitized' : round(precision[1], 2),
'Precision Uninfected' : round(precision[0], 2),
'Recall Parasitized' : round(recall[1], 2),
'Recall Uninfected' : round(recall[0], 2),
'F1 Score Parasitized' : round(f1_score[1], 2),
'F1 Score Uninfected' : round(f1_score[0], 2)
}, ignore_index=True)
# print out the performances of models created so far
model_performances
| Model | Accuracy | Precision Parasitized | Precision Uninfected | Recall Parasitized | Recall Uninfected | F1 Score Parasitized | F1 Score Uninfected | |
|---|---|---|---|---|---|---|---|---|
| 0 | Base CNN Model | 0.87 | 0.81 | 0.95 | 0.96 | 0.78 | 0.88 | 0.86 |
| 1 | Using more layers | 0.94 | 0.95 | 0.93 | 0.93 | 0.95 | 0.94 | 0.94 |
| 2 | Using Batch Normalization | 0.87 | 0.99 | 0.80 | 0.75 | 1.00 | 0.85 | 0.89 |
| 3 | Using HSV images | 0.94 | 0.94 | 0.94 | 0.94 | 0.94 | 0.94 | 0.94 |
# Clear Keras' backend to clear the history of previous models
keras_backend.clear_session()
# Re-init random numbers
random.seed(15)
np.random.seed(15)
tf.random.set_seed(15)
# Create the image data generator using Keras
from keras.preprocessing.image import ImageDataGenerator
data_generator = ImageDataGenerator(rotation_range=45, # rotate images randomly upto 45 degrees
zoom_range=0.2, # zoom the image upto 20%
horizontal_flip=True, # flip the image horizontally
vertical_flip=True, # and verticaly, randomly
fill_mode='nearest')
# Visualize a sampling of 18 augmented images
aug_images = []
data_iterator = data_generator.flow(X_train_normalized, y_train_encoded, batch_size=1)
for i in range(18):
aug_image = data_iterator.next()[0][0, :]
aug_images.append(aug_image)
aug_images = np.array(aug_images)
fig = plt.figure(figsize=(12, 6)) # Defining the figure size to 12x6
rows = 3 # number of rows in the plot
cols = 6 # number of columns in the plot
for i in range(18): # Iterate through rows first
ax = fig.add_subplot(rows, cols, i+1) # Adding subplot
ax.imshow(aug_images[i, :], cmap=plt.get_cmap('gray')) # Plotting the image using cmap=gray
plt.xticks([]) # Remove un-necessary ticks on the X axis
plt.yticks([]) # Remove un-necessary ticks on the y axis
plt.show()
# We can use the same architecture as the last model we built to use with augmented training images
model_augmented = Sequential()
model_augmented.add(Conv2D(16, (3, 3), padding="same", input_shape=(64, 64, 3)))
model_augmented.add(LeakyReLU(alpha=0.1))
model_augmented.add(Conv2D(32, (3, 3), padding="same"))
model_augmented.add(LeakyReLU(alpha=0.1))
model_augmented.add(MaxPooling2D(pool_size=(2, 2)))
model_augmented.add(BatchNormalization())
model_augmented.add(Conv2D(32, (3, 3), padding="same"))
model_augmented.add(LeakyReLU(alpha=0.1))
model_augmented.add(Conv2D(64, (3, 3), padding="same"))
model_augmented.add(LeakyReLU(alpha=0.1))
model_augmented.add(MaxPooling2D(pool_size=(2, 2)))
model_augmented.add(BatchNormalization())
model_augmented.add(Flatten())
model_augmented.add(Dense(32))
model_augmented.add(LeakyReLU(alpha=0.1))
model_augmented.add(Dropout(0.5))
model_augmented.add(Dense(2, activation='softmax'))
model_augmented.compile(loss='categorical_crossentropy', optimizer=Adam(learning_rate=0.001), metrics=['accuracy'])
early_stopping_cb = tf.keras.callbacks.EarlyStopping(
monitor='accuracy', min_delta=0.1, patience=3, verbose=1, restore_best_weights=True)
# Prep the image iterators
# Generate augmented images from in-memory images of training dataset
data_iterator = data_generator.flow(X_train_normalized, y_train_encoded, batch_size=32)
# Generate validation images from test dataset.
# We are not applying any transformation to these validation images, so that we can validate against real data
val_generator = ImageDataGenerator()
val_iterator = val_generator.flow(X_test_normalized, y_test_encoded, batch_size=32)
# Train the model using prepared iterators that use training and test images
model_history = model_augmented.fit(data_iterator, steps_per_epoch=len(X_train_normalized) // 32,
epochs=50, callbacks=[early_stopping_cb],
validation_data = val_iterator,
validation_steps=X_test_normalized.shape[0] // 32
)
Epoch 1/50 779/779 [==============================] - 29s 34ms/step - loss: 0.4903 - accuracy: 0.8198 - val_loss: 0.1380 - val_accuracy: 0.9537 Epoch 2/50 779/779 [==============================] - 26s 33ms/step - loss: 0.1632 - accuracy: 0.9518 - val_loss: 0.0718 - val_accuracy: 0.9826 Epoch 3/50 779/779 [==============================] - 26s 33ms/step - loss: 0.1289 - accuracy: 0.9620 - val_loss: 0.2518 - val_accuracy: 0.9375 Epoch 4/50 779/779 [==============================] - 26s 33ms/step - loss: 0.1151 - accuracy: 0.9639 - val_loss: 0.0871 - val_accuracy: 0.9730 Epoch 5/50 779/779 [==============================] - ETA: 0s - loss: 0.1085 - accuracy: 0.9667Restoring model weights from the end of the best epoch: 2. 779/779 [==============================] - 26s 33ms/step - loss: 0.1085 - accuracy: 0.9667 - val_loss: 0.0526 - val_accuracy: 0.9819 Epoch 5: early stopping
# Test the accuracy against test data
test_accuracy = model_augmented.evaluate(X_test_normalized, y_test_encoded, verbose=2)
82/82 - 0s - loss: 0.0717 - accuracy: 0.9827 - 344ms/epoch - 4ms/step
# Plot the accuracy of training and validation sets for the model that used augmented images
plotAccuracy(model_history, 'Model using Augmented images')
# Predict the labels for the test dataset
y_pred = model_augmented.predict(X_test_normalized) # returns prediction probabilities for each label
# Convert the prediction probablities back to single labels
y_pred_labels = np.argmax(y_pred, axis=1)
y_test_labels = np.argmax(y_test_encoded, axis=1)
metrics = classification_report(y_test_labels, y_pred_labels)
print(metrics)
82/82 [==============================] - 0s 3ms/step
precision recall f1-score support
0 0.98 0.98 0.98 1300
1 0.98 0.98 0.98 1300
accuracy 0.98 2600
macro avg 0.98 0.98 0.98 2600
weighted avg 0.98 0.98 0.98 2600
# Print the Confusion Matrix for test predictions
confusion_matrix = tf.math.confusion_matrix(y_test_labels, y_pred_labels)
_, ax = plt.subplots(figsize=(5, 4))
sns.heatmap(confusion_matrix, annot=True, linewidths=.4, fmt="d", square=True, ax=ax)
<Axes: >
# Collect the performance metrics for a later comparison of various models
precision, recall, f1_score, _ = precision_recall_fscore_support(y_test_labels, y_pred_labels)
accuracy = float(metrics.split()[-2])
model_performances = model_performances.append({
'Model' : 'Using augmented images',
'Accuracy' : accuracy,
'Precision Parasitized' : round(precision[1], 2),
'Precision Uninfected' : round(precision[0], 2),
'Recall Parasitized' : round(recall[1], 2),
'Recall Uninfected' : round(recall[0], 2),
'F1 Score Parasitized' : round(f1_score[1], 2),
'F1 Score Uninfected' : round(f1_score[0], 2)
}, ignore_index=True)
# print out the performances of models created so far
model_performances
| Model | Accuracy | Precision Parasitized | Precision Uninfected | Recall Parasitized | Recall Uninfected | F1 Score Parasitized | F1 Score Uninfected | |
|---|---|---|---|---|---|---|---|---|
| 0 | Base CNN Model | 0.87 | 0.81 | 0.95 | 0.96 | 0.78 | 0.88 | 0.86 |
| 1 | Using more layers | 0.94 | 0.95 | 0.93 | 0.93 | 0.95 | 0.94 | 0.94 |
| 2 | Using Batch Normalization | 0.87 | 0.99 | 0.80 | 0.75 | 1.00 | 0.85 | 0.89 |
| 3 | Using HSV images | 0.94 | 0.94 | 0.94 | 0.94 | 0.94 | 0.94 | 0.94 |
| 4 | Using augmented images | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 |
Now, let us try to use a pretrained model like VGG16 and check how it performs on our data.
The previous model used only the augmented images for training and validation. Let us use both the original images and augmented images, as it might perform even better.
# Clear Keras' backend to clear the history of previous models
keras_backend.clear_session()
# Re-init random numbers
random.seed(18)
np.random.seed(18)
tf.random.set_seed(18)
# Let's use both original dataset along with the augmented images generated
# we already have a ImageDataGenerator for augmented images, we will re-use it here
# Create a genearotr for original images
original_generator = ImageDataGenerator()
# collect flow'ed images from both original and augmented/generated images
X_train_original = original_generator.flow(X_train_normalized, y_train_encoded, batch_size=32)
X_train_augmented = data_generator.flow(X_train_normalized, y_train_encoded, batch_size=32)
# concatenate the generaotrs
def CombineGenerators(gen1, gen2):
while True:
x1, y1 = gen1.next()
x2, y2 = gen2.next()
yield (np.concatenate((x1, x2), axis=0), np.concatenate((y1, y2), axis=0))
X_train_combined = CombineGenerators(X_train_original, X_train_augmented)
# We can use the same architecture as the last model we built to use with augmented training images
model_orig_aug = Sequential()
model_orig_aug.add(Conv2D(16, (3, 3), padding="same", input_shape=(64, 64, 3)))
model_orig_aug.add(LeakyReLU(alpha=0.1))
model_orig_aug.add(Conv2D(32, (3, 3), padding="same"))
model_orig_aug.add(LeakyReLU(alpha=0.1))
model_orig_aug.add(MaxPooling2D(pool_size=(2, 2)))
model_orig_aug.add(BatchNormalization())
model_orig_aug.add(Conv2D(32, (3, 3), padding="same"))
model_orig_aug.add(LeakyReLU(alpha=0.1))
model_orig_aug.add(Conv2D(64, (3, 3), padding="same"))
model_orig_aug.add(LeakyReLU(alpha=0.1))
model_orig_aug.add(MaxPooling2D(pool_size=(2, 2)))
model_orig_aug.add(BatchNormalization())
model_orig_aug.add(Flatten())
model_orig_aug.add(Dense(32))
model_orig_aug.add(LeakyReLU(alpha=0.1))
model_orig_aug.add(Dropout(0.5))
model_orig_aug.add(Dense(2, activation='softmax'))
model_orig_aug.compile(loss='categorical_crossentropy', optimizer=Adam(learning_rate=0.001), metrics=['accuracy'])
# Train the model using prepared iterators that use training and test images
model_history = model_orig_aug.fit(X_train_combined,
steps_per_epoch=len(X_train_normalized) * 2 // 32,
epochs=50, callbacks=[early_stopping_cb],
validation_data = val_iterator,
validation_steps=X_test_normalized.shape[0] // 32
)
Epoch 1/50 1559/1559 [==============================] - 59s 36ms/step - loss: 0.2557 - accuracy: 0.9064 - val_loss: 0.0605 - val_accuracy: 0.9865 Epoch 2/50 1559/1559 [==============================] - 55s 35ms/step - loss: 0.0902 - accuracy: 0.9714 - val_loss: 0.0673 - val_accuracy: 0.9807 Epoch 3/50 1559/1559 [==============================] - 55s 35ms/step - loss: 0.0797 - accuracy: 0.9746 - val_loss: 0.0467 - val_accuracy: 0.9869 Epoch 4/50 1559/1559 [==============================] - ETA: 0s - loss: 0.0755 - accuracy: 0.9756Restoring model weights from the end of the best epoch: 1. 1559/1559 [==============================] - 55s 35ms/step - loss: 0.0755 - accuracy: 0.9756 - val_loss: 0.0467 - val_accuracy: 0.9846 Epoch 4: early stopping
# Test the accuracy against test data
test_accuracy = model_orig_aug.evaluate(X_test_normalized, y_test_encoded, verbose=2)
82/82 - 0s - loss: 0.0603 - accuracy: 0.9865 - 330ms/epoch - 4ms/step
# Plot the accuracy of training and validation sets for the model that used augmented images
plotAccuracy(model_history, 'Model using Origianal + Augmented images')
# Predict the labels for the test dataset
y_pred = model_orig_aug.predict(X_test_normalized) # returns prediction probabilities for each label
# Convert the prediction probablities back to single labels
y_pred_labels = np.argmax(y_pred, axis=1)
y_test_labels = np.argmax(y_test_encoded, axis=1)
metrics = classification_report(y_test_labels, y_pred_labels)
print(metrics)
82/82 [==============================] - 0s 3ms/step
precision recall f1-score support
0 0.98 0.99 0.99 1300
1 0.99 0.98 0.99 1300
accuracy 0.99 2600
macro avg 0.99 0.99 0.99 2600
weighted avg 0.99 0.99 0.99 2600
# Print the Confusion Matrix for test predictions
confusion_matrix = tf.math.confusion_matrix(y_test_labels, y_pred_labels)
_, ax = plt.subplots(figsize=(5, 4))
sns.heatmap(confusion_matrix, annot=True, linewidths=.4, fmt="d", square=True, ax=ax)
<Axes: >
# Collect the performance metrics for a later comparison of various models
precision, recall, f1_score, _ = precision_recall_fscore_support(y_test_labels, y_pred_labels)
accuracy = float(metrics.split()[-2])
model_performances = model_performances.append({
'Model' : 'Using original AND augmented images',
'Accuracy' : accuracy,
'Precision Parasitized' : round(precision[1], 2),
'Precision Uninfected' : round(precision[0], 2),
'Recall Parasitized' : round(recall[1], 2),
'Recall Uninfected' : round(recall[0], 2),
'F1 Score Parasitized' : round(f1_score[1], 2),
'F1 Score Uninfected' : round(f1_score[0], 2)
}, ignore_index=True)
# print out the performances of models created so far
model_performances
| Model | Accuracy | Precision Parasitized | Precision Uninfected | Recall Parasitized | Recall Uninfected | F1 Score Parasitized | F1 Score Uninfected | |
|---|---|---|---|---|---|---|---|---|
| 0 | Base CNN Model | 0.87 | 0.81 | 0.95 | 0.96 | 0.78 | 0.88 | 0.86 |
| 1 | Using more layers | 0.94 | 0.95 | 0.93 | 0.93 | 0.95 | 0.94 | 0.94 |
| 2 | Using Batch Normalization | 0.87 | 0.99 | 0.80 | 0.75 | 1.00 | 0.85 | 0.89 |
| 3 | Using HSV images | 0.94 | 0.94 | 0.94 | 0.94 | 0.94 | 0.94 | 0.94 |
| 4 | Using augmented images | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 |
| 5 | Using original AND augmented images | 0.99 | 0.99 | 0.98 | 0.98 | 0.99 | 0.99 | 0.99 |
# Clear Keras' backend to clear the history of previous models
keras_backend.clear_session()
# Re-init random numbers
random.seed(21)
np.random.seed(21)
tf.random.set_seed(21)
# Use the pre-trained model VGG16 without the top layers
from keras.applications import VGG16
vgg_pretrained = VGG16(weights='imagenet', # use weights that are pre-trained on ImageNet
include_top=False, # do not include 3 fully connected layers at the top of network
input_shape=(64, 64, 3)) # image shape for the input layer
for layer in vgg_pretrained.layers:
layer.trainable = False # no need to train the pre-trained layers
model_vgg_based = Sequential() # this is our model that will make use of pre-trained VGG16
model_vgg_based.add(vgg_pretrained) # add the pre-trained VGG16 as a layer to our model
# Add our own fully connected layers
model_vgg_based.add(Flatten())
model_vgg_based.add(Dense(32))
model_vgg_based.add(LeakyReLU(alpha=0.1))
model_vgg_based.add(Dense(2, activation='softmax'))
# Compile our model that uses transfer learning
model_vgg_based.compile(loss='categorical_crossentropy', optimizer=Adam(learning_rate=0.001), metrics=['accuracy'])
# Let's define a callback to enable early stopping, if no improvement between successive epochs
early_stopping_cb = tf.keras.callbacks.EarlyStopping(
monitor='val_accuracy', min_delta=0.05, patience=3, verbose=1, restore_best_weights=True)
# Train the model that uses learnings from VGG16
model_history = model_vgg_based.fit(X_train_normalized, y_train_encoded, validation_split=0.2, batch_size=32,
verbose=1, epochs=50, callbacks=[early_stopping_cb])
Epoch 1/50 624/624 [==============================] - 14s 20ms/step - loss: 0.1869 - accuracy: 0.9253 - val_loss: 0.3552 - val_accuracy: 0.8644 Epoch 2/50 624/624 [==============================] - 11s 17ms/step - loss: 0.1362 - accuracy: 0.9495 - val_loss: 0.2502 - val_accuracy: 0.9004 Epoch 3/50 624/624 [==============================] - 11s 18ms/step - loss: 0.1299 - accuracy: 0.9495 - val_loss: 0.2677 - val_accuracy: 0.8972 Epoch 4/50 621/624 [============================>.] - ETA: 0s - loss: 0.1234 - accuracy: 0.9532Restoring model weights from the end of the best epoch: 1. 624/624 [==============================] - 11s 18ms/step - loss: 0.1233 - accuracy: 0.9533 - val_loss: 0.3400 - val_accuracy: 0.8590 Epoch 4: early stopping
# Test the accuracy against test data
test_accuracy = model_vgg_based.evaluate(X_test_normalized, y_test_encoded, verbose=2)
82/82 - 2s - loss: 0.2535 - accuracy: 0.9008 - 2s/epoch - 20ms/step
# Plot the accuracy of training and validation sets for the model that used augmented images
plotAccuracy(model_history, 'Model using VGG16 transfer learning')
# Predict the labels for the test dataset
y_pred = model_vgg_based.predict(X_test_normalized) # returns prediction probabilities for each label
# Convert the prediction probablities back to single labels
y_pred_labels = np.argmax(y_pred, axis=1)
y_test_labels = np.argmax(y_test_encoded, axis=1)
metrics = classification_report(y_test_labels, y_pred_labels)
print(metrics)
82/82 [==============================] - 1s 13ms/step
precision recall f1-score support
0 0.98 0.82 0.89 1300
1 0.84 0.98 0.91 1300
accuracy 0.90 2600
macro avg 0.91 0.90 0.90 2600
weighted avg 0.91 0.90 0.90 2600
# Print the Confusion Matrix for test predictions
confusion_matrix = tf.math.confusion_matrix(y_test_labels, y_pred_labels)
_, ax = plt.subplots(figsize=(5, 4))
sns.heatmap(confusion_matrix, annot=True, linewidths=.4, fmt="d", square=True, ax=ax)
plt.show()
# Collect the performance metrics for a later comparison of various models
precision, recall, f1_score, _ = precision_recall_fscore_support(y_test_labels, y_pred_labels)
accuracy = float(metrics.split()[-2])
model_performances = model_performances.append({
'Model' : 'Transfer from VGG16',
'Accuracy' : accuracy,
'Precision Parasitized' : round(precision[1], 2),
'Precision Uninfected' : round(precision[0], 2),
'Recall Parasitized' : round(recall[1], 2),
'Recall Uninfected' : round(recall[0], 2),
'F1 Score Parasitized' : round(f1_score[1], 2),
'F1 Score Uninfected' : round(f1_score[0], 2)
}, ignore_index=True)
# print out the performances of models created so far
model_performances
| Model | Accuracy | Precision Parasitized | Precision Uninfected | Recall Parasitized | Recall Uninfected | F1 Score Parasitized | F1 Score Uninfected | |
|---|---|---|---|---|---|---|---|---|
| 0 | Base CNN Model | 0.87 | 0.81 | 0.95 | 0.96 | 0.78 | 0.88 | 0.86 |
| 1 | Using more layers | 0.94 | 0.95 | 0.93 | 0.93 | 0.95 | 0.94 | 0.94 |
| 2 | Using Batch Normalization | 0.87 | 0.99 | 0.80 | 0.75 | 1.00 | 0.85 | 0.89 |
| 3 | Using HSV images | 0.94 | 0.94 | 0.94 | 0.94 | 0.94 | 0.94 | 0.94 |
| 4 | Using augmented images | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 |
| 5 | Using original AND augmented images | 0.99 | 0.99 | 0.98 | 0.98 | 0.99 | 0.99 | 0.99 |
| 6 | Transfer from VGG16 | 0.90 | 0.84 | 0.98 | 0.98 | 0.82 | 0.91 | 0.89 |
# Plot the performance with all metrics, for all models we built, so that we can compare them visually
perf_comparison = model_performances.copy(deep=True) # make a copy, so that original dataframe can be left intact
perf_comparison.set_index('Model', inplace=True)
perf_comparison.plot(kind='bar', figsize=(10, 5))
plt.title('Compare performance of models we built')
plt.xlabel('Model')
plt.ylabel('Score')
plt.legend(bbox_to_anchor=(1.30, 1.0), loc='upper right')
plt.show()
Think about it:
# Choose final model to recommend
final_model = model_orig_aug
Recommendations
Cost vs Benefits
Risks and Challenges
Improvements that can be done:
Can the model performance be improved using other pre-trained models or different CNN architecture?
What are the most meaningful insights from the data relevant to the problem?